Understanding the Social Context of Eating with Multimodal Smartphone Sensing: The Role of Country Diversity

Understanding the social context of eating is crucial for promoting healthy eating behaviors by providing timely interventions. Multimodal smartphone sensing data has the potential to provide valuable insights into eating behavior, particularly in mobile food diaries and mobile health applications. However, research on the social context of eating with smartphone sensor data is limited, despite extensive study in nutrition and behavioral science. Moreover, the impact of country differences on the social context of eating, as measured by multimodal phone sensor data and self-reports, remains under-explored. To address this research gap, we present a study using a smartphone sensing dataset from eight countries (China, Denmark, India, Italy, Mexico, Mongolia, Paraguay, and the UK). Our study focuses on a set of approximately 24K self-reports on eating events provided by 678 college students to investigate the country diversity that emerges from smartphone sensors during eating events for different social contexts (alone or with others). Our analysis revealed that while some smartphone usage features during eating events were similar across countries, others exhibited unique behaviors in each country. We further studied how user and country-specific factors impact social context inference by developing machine learning models with population-level (non-personalized) and hybrid (partially personalized) experimental setups. We showed that models based on the hybrid approach achieve AUC scores up to 0.75 with XGBoost models. These findings have implications for future research on mobile food diaries and mobile health sensing systems, emphasizing the importance of considering country differences in building and deploying machine learning models to minimize biases and improve generalization across different populations

Understanding Social Context from Smartphone Sensing: Generalization Across Countries and Daily Life Moments

Understanding and longitudinally tracking the social context of people help in understanding their behavior and mental well-being better. Hence, instead of burdensome questionnaires, some studies used passive smartphone sensors to infer social context with machine learning models. However, the few studies that have been done up to date have focused on unique, situated contexts (i.e., when eating or drinking) in one or two countries, hence limiting the understanding of the inference in terms of generalization to (i) everyday life occasions and (ii) different countries. In this paper, we used a novel, large-scale, and multimodal smartphone sensing dataset with over 216K self-reports collected from over 580 participants in five countries (Mongolia, Italy, Denmark, UK, Paraguay), first to understand whether social context inference (i.e., alone or not) is feasible with sensor data, and then, to know how behavioral and country-level diversity affects the inference. We found that (i) sensor features from modalities such as activity, location, app usage, Bluetooth, and WiFi could be informative of social context; (ii) partially personalized multi-country models (trained and tested with data from all countries) and country-specific models (trained and tested within countries) achieved similar accuracies in the range of 80%-90%; and (iii) models do not generalize well to unseen countries regardless of geographic similarity

Inferring accurate bus trajectories from noisy estimated arrival time records

National Research Foundation (NRF) Singapore under its International Research Centres in Singapore Funding Initiativ

Jointly optimizing sensing pipelines for multimodal mixed reality interaction

National Research Foundation (NRF) Singapore under International Research Centres in Singapore Funding Initiative; Ministry of Education, Singapore under its Academic Research Funding Tier

BuSCOPE: Fusing individual & aggregated mobility behavior for “Live” smart city services

While analysis of urban commuting data has a long and demonstrated history of providing useful insights into human mobility behavior, such analysis has been performed largely in offline fashion and to aid medium-to-long term urban planning. In this work, we demonstrate the power of applying predictive analytics on real-time mobility data, specifically the smart-card generated trip data of millions of public bus commuters in Singapore, to create two novel and "live" smart city services. The key analytical novelty in our work lies in combining two aspects of urban mobility: (a) conformity: which reflects the predictability in the aggregated flow of commuters along bus routes, and (b) regularity: which captures the repeated trip patterns of each individual commuter. We demonstrate that the fusion of these two measures of behavior can be performed at city-scale using our BuScope platform, and can be used to create two innovative smart city applications. The Last-Mile Demand Generator provides O(mins) lookahead into the number of disembarking passengers at neighborhood bus stops; it achieves over 85% accuracy in predicting such disembarkations by an ingenious combination of individual-level regularity with aggregate-level conformity. By moving driverless vehicles proactively to match this predicted demand, we can reduce wait times for disembarking passengers by over 75%. Independently, the Neighborhood Event Detector uses outlier measures of currently operating buses to detect and spatiotemporally localize dynamic urban events, as much as 1.5 hours in advance, with a localization error of 450 meters.Comment: ACM MobiSys 201

Quantified Canine: Inferring Dog Personality From Wearables

Being able to assess dog personality can be used to, for example, match shelter dogs with future owners, and personalize dog activities. Such an assessment typically relies on experts or psychological scales administered to dog owners, both of which are costly. To tackle that challenge, we built a device called "Patchkeeper" that can be strapped on the pet's chest and measures activity through an accelerometer and a gyroscope. In an in-the-wild deployment involving 12 healthy dogs, we collected 1300 hours of sensor activity data and dog personality test results from two validated questionnaires. By matching these two datasets, we trained ten machine-learning classifiers that predicted dog personality from activity data, achieving AUCs in [0.63-0.90], suggesting the value of tracking the psychological signals of pets using wearable technologies.Comment: 26 pages, 9 figures, 4 table

Inferring Mood-While-Eating with Smartphone Sensing and Community-Based Model Personalization

The interplay between mood and eating has been the subject of extensive research within the fields of nutrition and behavioral science, indicating a strong connection between the two. Further, phone sensor data have been used to characterize both eating behavior and mood, independently, in the context of mobile food diaries and mobile health applications. However, limitations within the current body of literature include: i) the lack of investigation around the generalization of mood inference models trained with passive sensor data from a range of everyday life situations, to specific contexts such as eating, ii) no prior studies that use sensor data to study the intersection of mood and eating, and iii) the inadequate examination of model personalization techniques within limited label settings, as we commonly experience in mood inference. In this study, we sought to examine everyday eating behavior and mood using two datasets of college students in Mexico (N_mex = 84, 1843 mood-while-eating reports) and eight countries (N_mul = 678, 329K mood reports incl. 24K mood-while-eating reports), containing both passive smartphone sensing and self-report data. Our results indicate that generic mood inference models decline in performance in certain contexts, such as when eating. Additionally, we found that population-level (non-personalized) and hybrid (partially personalized) modeling techniques were inadequate for the commonly used three-class mood inference task (positive, neutral, negative). Furthermore, we found that user-level modeling was challenging for the majority of participants due to a lack of sufficient labels and data from the negative class. To address these limitations, we employed a novel community-based approach for personalization by building models with data from a set of similar users to a target user

Smartphone Sensing for the Well-being of Young Adults: A Review

Over the years, mobile phones have become versatile devices with a multitude of capabilities due to the plethora of embedded sensors that enable them to capture rich data unobtrusively. In a world where people are more conscious regarding their health and well-being, the pervasiveness of smartphones has enabled researchers to build apps that assist people to live healthier lifestyles, and to diagnose and monitor various health conditions. Motivated by the high smartphone coverage among young adults and the unique issues they face, in this review paper, we focus on studies that have used smartphone sensing for the well-being of young adults. We analyze existing work in the domain from two perspectives, namely Data Perspective and System Perspective. For both these perspectives, we propose taxonomies motivated from human science literature, which enable to identify important study areas. Furthermore, we emphasize the importance of diversity-awareness in smartphone sensing, and provide insights and future directions for researchers in ubiquitous and mobile computing, and especially to new researchers who want to understand the basics of smartphone sensing research targeting the well-being of young adults